Identifying and modelling delay feedback systems 
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Systems with delayed feedback can possess chaotic attractors with extremely high dimension, 
even if only a few physical degrees of freedom are involved. We propose a state space reconstruction 
from time series data of a scalar observable, coming along with a novel method to identify and 
model such systems, if a single variable is fed back. Making use of special properties of the feedback 
structure, we can understand the structure of the system by constructing equivalent equations of 
motion in spaces with dimensions which can be much smaller than the dimension of the chaotic 
attractor. We verify our method using both numerical and experimental data. 



Since it is known that simple nonlinear dynamical sys- 
tems can produce extremely complicated, aperiodic mo- 
tion, the search for low-dimensional deterministic struc- 
tures in observed time series has become a common ap- 
proach to the understanding of complex time dependent 
phenomena. The methods, summarised under the term 
non-linear time series analysis, have meanwhile acquired 
a high standard It is possible, in spite of problems 

with noise or non-stationarity, to extract estimates of the 
fractal dimensions, the entropy and the Lyapunov expo- 
nents from observed data. Often, the ultimate goal is the 
construction of model-equations from the data. 

As a first step of this kind of data analysis, a state 
space has to be reconstructed from the data, since usu- 
ally only a single observable is measured (scalar data) 
1^. The mathematical basis for this reconstruction is 
given by the delay embedding theorems of Takens and 
Sauer et al. The theorem, in the formulation of the 
latter authors, states that, loosely speaking, forming m- 
dimensional delay vectors Vi = {xi,Xi-i, . . . ,Xi-m+i) of 
the sequence of measurements {xj} yields a one to one 
description of the attractor of the system, if m > 21?/, 
where -D/ is the fractal dimension of the attractor. If 
Df is larger than about 5, this method usually fails in 
practice, since the number N of observed measurements 
covers the high dimensional attractor insufficiently. Ad- 
ditionally, the time delay embedding method introduces 
distortions for data with large Kolmogorov-Sinai en- 
tropy, such that the typical minimal inter-point-distance 
which can be reached by reasonably large data sets (say, 
N = 10^) is still too large for a detection of the deter- 
ministic structure Q. 



A special class of dynamical systems is that of the de- 
layed feedback systems [Q. Despite a small number of 
physical variables, their phase space is infinite dimen- 
sional, namely the space of all differentiable functions on 
the time interval [0, tq], where tq is the delay time of 
the feedback. They can thus possess chaotic attractors 
of arbitrarily high dimension. The direct reconstruction 
of the attractor by the time delay embedding method is 
therefore usually impossible. Recently a method 

was proposed to identify such systems from data when 
they involve only a single variable, i.e. when they can be 
described by a scalar delay-differential equation 
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The structure of the evolution equation (^ is obvi- 
ously independent of the dimension of the resulting at- 
tractor. Indeed, in the space spanned by the vectors 
{x{t),x{t),x{t~T)),Eq. d) defines a constraint if t = tq 
and the data collapse onto a two dimensional manifold, 
deHnedhy x{t) — f(x{t),x{t — To)) = 0. This property was 
used in |^,^ to identify these systems and to recover the 
delay time tq and the function /. Similar results were 
achieved for d-dimensional systems if all d components 
are measured @. 

In this paper we introduce a nontrivial generalisation 
for delay systems with more than one variable in the real- 
istic situation that only a single observable of the system 
was measured. This method allows one to reconstruct 
state vectors with which we can determine the time de- 
lay To and an equation of motion. We will thus be able to 
perform predictions and to model the system. The idea 
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of our method is to combine a Takens-like embedding 
technique with the structure of time delayed feedback 
systems, and to use the abihty of making forecasts as a 
criterion for a successful reconstruction of a state space. 
Thus the state space reconstruction implies the deriva- 
tion of evolution equations in this space. 

We start with the one-dimensional case (|l|). In the 
following, To will be the time delay of the feedback mea- 
sured in multiples of the sampling interval At and for 
simplicity will be assumed to be integer, whereas t will 
be a trial value used in the reconstruction. A successful 
reconstruction will lead to r = tq. We make the ansatz 



(2) 



where Xi is an estimate for the time derivative computed 
from differences of the time series data {xj}. For g.i we 
choose a local linear model giivi) — bi + a.iVi, where the 
parameters hi and Oj depend on the spatial coordinates 
Vi — {xi, Xi-r)- This model represents the first terms of a 
Taylor series expansion of the unknown global nonlinear 
model around Vi. The parameters can be obtained for 
each time index j by the local least squares fit 
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where Uj is a sufficiently small neighbourhood of Vj = 
{xj,Xj-r) and Njj^ its cardinal number. The correct un- 
known value To of Eq.(l) is recovered if (j(t) = (o'j(T))j 
is at its absolute minimum, since only for this value of r 
all triples {xi, Xi, Xi-r) fulfil simultaneously a constraint, 
if the motion is chaotic. 

Figure |l| shows the relative forecast errors for the 
Mackey-Glass |11 system 



ax{t — To) 
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bx{t) , 
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with a = 0.1, 6 = 0.2 and tq — 100. The figure demon- 
strates that the correct delay time is detected easily. 
Once we know tq , we can determine gj to iterate the sys- 
tem by a simple Euler integration Xj+i = Xj + Atgj{vj), 
provided the numerical trajectory remains on the attrac- 
tor formed by the observed data. It is plausible that one 
could as well replace the local linear models gi by a single 
global nonlinear function g{v) = ^ ajfj{v), e.g. a bivari- 
ate polynomial or radial basis functions, by minimising 
X]iL2(^i+i ~ Si o,jfj{xi,Xi-r))^ with respect to the co- 



efficients 



which is standard in nonlinear function fit- 



ting. We used the radial basis functions successfully for 
the Mackey-Glass data discussed above. 

In general one has to expect a delayed feedback system 
to involve several variables. For the relevant situation 
that only one of these variables enters the feedback loop 
the equation of motion reads 



m^f{m,y'{t-ro)) 
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where y G R'^ and y^{t — tq) is the single component 
which is fed back into the system. In many laboratory 
experiments with time delayed feedback exactly this is 
the case. If we again possess only a scalar time series, we 
have to recover the unobserved variables of the system 
in addition to determining the delay time tq. We can do 
this by exploiting the ideas of Casdagli for input -output 
systems [ p^ . 

Consider a non-autonomous deterministic system with 
an arbitrary input et (which could even be noise) of the 
form 



yt+i = .fiyt,et) 



(6) 



where y G R''. If we possess a scalar time series of one 
component of y, a standard time delay reconstruction 
cannot work, since each additional measurement contains 
an additional uncertainty, namely ej. If one simultane- 
ously measures the input, one can form delay vectors 



«t = {Vt,yt-i, ■ ■ ■ ,yt~-'m+i,et, 
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Casdagli argues that in this space the deterministic dy- 
namics can be reconstructed, if to = 2d, generically [^ . 

In our case the delayed feedback variable can be inter- 
preted as non-autonomous input. Following Casdagli's 
reasoning, we have to include these inputs in the delay 
vector. 
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Here it is assumed that x is identical to or a unique func- 
tion of only the delayed variable from Eq. (^), and toi is 
the dimension we need to fully determine the actual state 
of the system, while TO2 is the number of inputs required. 
In general we expect m2 = toi. Due to strong correla- 
tions between successive measurements TO2 < TOi could 
be sufficient. Following Casdagli's results we expect that 
TOi = TO2 = 2c? is sufficient for a unique representation of 
{y\t),y\t - t)), t = iAt, of Eq. (||). Note that the frac- 
tal dimension of the attractor can be much larger than 
Ad and that, in particular, the embedding we propose is 
independent of the attractor dimension. 

In analogy to the 1-dimensional case we make an 
ansatz in this embedding space to recover the delay time 
and the dynamics of the d-dimensional system: 



= giiviir)) 
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where g here is again a local linear model, namely 
9i{vi{T)) — hi + diViir), and a successful reconstruction 
will lead to r = tq. 

For a demonstration on numerically generated data we 
use a generalisation of the Mackey-Glass equation intro- 
duced in pO| : 
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where we choose the parameters tobea = 3,p=1.5, 
ijj'^ = 1 and tq = 10. The Kaplan- Yorke dimension of this 
system is Dky ~ 13.5. The left panel of Fig. ^ shows its 
attractor in a two-dimensional representation. Figure |^ 
shows the one step prediction errors in an embedding 
space of the variable function of r. Already for an 

insufficient reconstruction mi = TO2 = 1 (upper curve) 
we see indications of the correct tq. If we increase toi and 
7712 to 2, the forecast errors decrease dramatically. At the 
correct value of t it is reduced by a factor of 20. Fur- 
ther increases of mi and m2 do not yield better forecast 
errors. 

This result can be verified analytically. An expression 
for the dynamics in delay space follows from rewriting 
Eq. ( p^ ) as second order in time 

±[t) = - uj'^x{t) - px{t) + uj'^f{x{t - To)) 

df{x{t - To)) 

+ —ru ^a;(t-ro), (11) 

ax(t — To) 

where / is the first term of the rhs of Eq. (pO|), and 
substituting derivatives by differences. Using a simple 
Euler-scheme for this substitution, two 'on-time' mea- 
surements and two delayed measurements are involved in 
the forecast, i.e. mi = TO2 = 2. The right panel of Fig. ^ 
shows data obtained by iterating the fitted dynamics g in 
Eq. (H) . The properties of the attractor are reproduced 
well, although the dimension of the delay space is less 
than a third of the estimated Kaplan- Yorke dimension. 

Experimental data from systems whose model equa- 
tions are not fully known are a challenge. We apply our 
state space reconstruction scheme to data from a CO2 
laser experiment with delayed feedback performed at the 
Istituto Nazionale di Ottica in Firence/Italy ||l^. The 
output power of the laser is measured, which is the vari- 
able which is electronically fed back into the system. Ex- 
perimental data are shown in the left panel of Fig. ^ in 
a representation which was proposed in iQ . The scalar 
time series {xi} is represented in a space-time plane by 
decomposing the time index i into an artificial discrete 
time n and a space variable s 



s + riTo with s G [0, to[ , 
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This presentation allows one to observe that the system 
behaves periodically with defects travelling through it. 
Applying our analysis in mi = m2 = 2 dimensions we 
obtain a delay of tq = 800, in perfect agreement with the 
value given by the experimentalists, and the forecast er- 
ror drops by a factor of « 10. Enlarging toi and m2 does 
not improve the results, whereas mi = m2 = 1 turns out 
to be insufficient. Measurement noise may here distort 
the estimates of the derivatives. The right panel of Fig. ^ 



shows the presentation of the time series obtained from 
the integrated fitted equation of motion. Comparing the 
structures visually we see a good agreement, differing 
details are even to be expected for a perfect model be- 
cause of the sensitive dependence on initial conditions. 
We thus conclude that the experimental data from the 
C02-laser are governed by a two-dimensional time-delay 
differential equation with a single delayed feedback vari- 
able. Therefore, we detect a considerable reduction of 
dimension in comparison to the six-dimensional model 
proposed in [|l5|, which has been derived from first prin- 
ciples. Our finding is in agreement with the conjecture 
of |l^ ] concerning the reduction of the dimension of a 
time-delay system close to a bifurcation. Without mak- 
ing use of the feedback structure, in standard embedding 
spaces with dimensions up to 6, we were unable to ob- 
tain models which reproduce the defect dynamics, and 
the optimal prediction error was much larger. 

When only a single quantity is measured in a system 
with time delayed feedback, the full state space is not 
accessible and also the "true" equations of motion. We 
proposed a novel method for the reconstruction of a space 
which is equivalent to the physical one. In this space 
equations of motion attain a very natural form, mirroring 
the delay structure of the unknown dynamical system. 
We thus work in a space whose dimension is usually much 
smaller than the attractor's dimension, although larger 
than the number of variables in the unknown space. We 
do not reconstruct the attractor by Takens's embedding, 
but only the unobserved variables and then exploit the 
special structure of the evolution equations. This is of 
particular interest when tq in the experiment is varied: 
The attractor dimension will vary, too, but the structure 
of our space remains invariant, and so do the equations 
of motion in this space. A successful reconstruction in- 
volves to detect the correct time delay tq, which is done 
here very efficiently by a minimisation of a kind of predic- 
tion error. A maximum of the auto-correlation function 
usually appears close to, but not exactly at tq ||i7|] . 

Although we only presented results for systems with 
a single delay time, our method can of course be easily 
generalised to systems with several delay times. Further- 
more, once we have modelled the system we can estimate 
the Lyapunov spectrum from this model. 

The problem of noise in experimental data will be sub- 
ject to future works. However, the successful treatment 
of the laser data shows that well controlled laboratory ex- 
periments provide data whose noise contamination does 
not harm the applicability of the concept. On the other 
hand it is clear that the forecast error cannot be smaller 
than the noise level, so that too strong noise could com- 
pletely destroy the detectability of the correct tq. 
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FIG. 1. One step prediction error for the Mackey-Glass system with to = 100. 



FIG. 2. One step prediction error for the two-dimensional generalized Mackey-Glass system with to = 10. 



FIG. 3. The attractor of the two dimensional Mackey Glass system. The left panel shows the original data, the right panel 
the data obtained by iterating the fitted dynamics using mi = m2 = 2. 



FIG. 4. Output power of a CO2 laser experiment. The left panel shows the original data, the right panel shows the data 
obtained from an iterated forecast in mi = m2 = 2 dimensions. The representation of the data is described in the text. 
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This figure "fig4a.jpg" is available in "jpg" format from: 



http://arXiv.org/ps/chao-dyn/9907019vl 



This figure "fig4b.jpg" is available in "jpg" format from: 
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